Method and apparatus for interactive annotation and measurement of time series data with automatic marker sequencing

ABSTRACT

A system displays time-series data, such as electrocardiographic data. The data may be displayed as a trace with markers identifying data features. The markers may be automatically sequenced by the system.

FIELD OF THE INVENTION

The invention relates to display systems and, more particularly, tointeractive displays for the presentation, annotation, and analysis ofthe features of electrocardiogram waveforms and other time series data.

BACKGROUND OF THE INVENTION

A number of interactive display devices have been developed to provideimmediate visual feedback to a user of a computer, or other electronicor electromechanical equipment, and to thereby allow the user toprecisely control operations related to the display. The movement of acursor on a display, for example, permits the user to insert wordswithin text as the user types a document. Similarly, by activating aslide bar at one side of the display, the user can scroll through thedocument much more rapidly. Pull-down and pop-up menus permit the userto activate other functions, such as saving or printing a document,checking the spelling of a document, etc. The display is interactive inthe sense that signals from an input device, such as a keyboard, amouse, touch-pad, touch-screen, or voice input system, is reflected inthe modification of the display and in an underlying modification ofdata related to the display. That is, for example, not only is thecursor moved on the screen in response to input from the mouse, theunderlying document, stored in electronic form within the computer, alsoreflects the cursor movement.

The real-time display provided by endoscopic instruments during surgeryallows a surgeon to precisely control the position, direction, and speedof surgical tools in the process of delicate brain surgery that wouldotherwise be impossible. Although the tool might be positionedphysically by powerful magnets, for example, control of the magnets andultimately of the surgical tools, is in the hands of a surgeon. Thesurgeon may rely upon a display to provide him with immediate visualfeedback via a live video feed while he employs a joystick or otherinput device to control the surgical tool. The surgeon may be operatingby “dead reckoning” in that the only feedback he may be receiving isfrom the video feed and from some sort of an indicator which reveals theposition of the tool within the patient, without revealing anythingabout the tool itself, that is, any changes in the cross-section of thetool, for example.

Interactive display devices are also used in the analysis of complexdata sets. Insight may be gained by viewing the data in a unique mannerthat permits the visual correlation of data. Just as Linneaus' binomialorganization of biological specimens into species and genus provided anorganizational framework for understanding the vast diversity of thebiosphere, an opportune display of data or of the results of operationsperformed on the data, may allow a user to gain insights that mightotherwise be overlooked.

Although current interactive displays provide adequate feedback for manyapplications, there is a need for an interactive display which providesvisual feedback to a user for operations that are more complex thansimply positioning a cursor within a field of text. In particular, thedisplay and on-screen measurement of time series data, such aselectrocardiogram data, would be highly desirable.

SUMMARY

An interactive display system in accordance with the principles of theinvention includes an input device, a controller, and a display. Thecontroller is configured to display time-series data, such aselectrocardiogram data, in graphical form, on the display. Thecontroller is also configured to position one or more markers on thedisplay, as dictated by user input received through an input device, andto correlate the underlying data to the point(s) indicated by the markerposition.

The controller may be responsive to signals from the input device bymodifying the size, shape, position, or other aspects of the marker.Such modifications to the marker are provided as a visual feedbackmechanism for a user. In addition to the marker modification, thecontroller may operate on data, or provide an indication to anothercontroller that the data should be operated upon, in a predeterminedmanner corresponding to the manipulation of the marker. For example, atime interval may be marked off by the manipulation of one or moremarkers, and the controller may respond, not only by positioning themarkers according to user input, but also by computing the time and/orother values (e.g., average signal level over the interval).Additionally, modifications to the position of the marker may bereflected by modifications to information displayed, with, for example,the coordinates of one or more markers displayed and updated “on thefly” as a marker is repositioned on the display. The coordinates mayrepresent a multi-dimensional space in which dimensions are devoted tosignal level, time, event number, or other variables, for example.Markers may be combined with other interactive display devices andtechniques such as pulldown or popup menus, or sliders, for example.

In an illustrative embodiment, the interactive display presentselectrocardiogram data in a manner that emulates the standard paperrecording format. For example, data from a standard ECG recording may bedisplayed as traces overlaid on a millimeter grid reference background.By displaying ECG data in much the same format as that of a conventionalECG paper printout, the system capitalizes on the pattern recognitionskills developed by cardiologists through years of training andexperience. Moreover, the interactive display system maintains theaspect ratio, thereby preserving the pattern-recognition advantages,during electronic magnification (zoom) operations. By preserving theaspect ratio in this manner, doubling, for example, both the horizontaland vertical scales for both the ECG waveform and the reference grid fora 2× magnified view of an ECG, a cardiologist may make precise,highly-refined ECG measurements, even while viewing an undistortedrepresentation of the ECG.

A user may select one or more features of interest by manipulating oneor more markers. Each marker may be a conventional marker, such as acursor such as is used in word-processing applications, for example.Alternately, a marker may be a vertical line that intersects thewaveform and has an associated alphanumeric character, such as a “P”(corresponding to atrial depolarization) to identify the meaning of themarked point. The marker may also be conic, such as a graphiccorresponding to a frontal plane QRS axis, for example. Intervals, suchas PR, QRS, QT, and RR intervals, may be measured on-screen using one ormore markers. That is, for example, a user may employ a single marker toindicate a point for which measurements, such as voltage and time, aredesired. The user may use a single marker to set the beginning of aninterval for which measurements are desired, then place a marker at theend of the desired interval, or a plurality of markers may be employedto denote data values or data intervals of interest.

Multiple markers may also be employed, with one assigned to mark thebeginning of an interval and one assigned to mark the end of an intervalof interest, for example. The interactive display system also providesfor “automatic” marking. For example, in one mode of operation a usermay mark the QRS onset feature on a trace of interest in response towhich the system completes the markings for the P (P onset), J (QRSend), and T (T wave end) points and computes the intervals associatedwith this beat. Measurements may be displayed in close proximity to themeasurement marks, overlaid on the strip chart background, and/or may bedisplayed in one or more separate “reporting areas on the display. Thedisplayed measurements may be updated continuously, so that, as a markeris moved across the strip chart background, the measurement displaycontinuously changes to reflect the updated position of the marker.Alternatively, the displayed measurements may be updated uponfinalization of a marker position. The finalization may be effectedthrough use of an “enter” keystroke, for example.

The time-series data may be obtained directly from an electrocardiogrammachine that provides digital output. If the electrocardiogram machineprovides only analog output, the analog signal(s) may be converted todigital form for processing by an interactive display in accordance withthe principles of the present invention. Whether the analog ECG signalsare converted to digital form by the ECG machine or in post-processing,a user may process the corresponding digital data on the interactivedisplay directly from the ECG machine or from stored ECG data.Additionally, one or more users may use an interactive display inaccordance with the principles of the present invention to process ECGdata that is obtained at one or more patient sites and transmitted, viaa telecommunications network, for example, to an ECG analysis center.Digitized ECG data may be transmitted to an analysis center and storedfor future processing or placed in a queue for immediate processing.Alarms of varying degrees of urgency may be activated by an interactivedisplay in accordance with the principles of the present invention inresponse to ECG data analyses.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and further features, aspects, and advantages of the inventionwill be apparent to those skilled in the art from the following detaileddescription, taken together with the accompanying drawings in which:

FIG. 1 is a conceptual block diagram of a system that may employ aninteractive display in accordance with the principles of the invention;

FIG. 2 is a conceptual block diagram of a system that may employ aremote data collection process through a telecommunications network inaccordance with the principles of the invention;

FIG. 3 is an illustrative display screen that contains a standard 12lead resting ECG in a standard configuration of 10 seconds by 12 leads.This screen is also used to illustrate some of the interactive featuresand display areas of a display in accordance with the principles of thepresent invention;

FIG. 4 is an illustrative display screen that contains another format ofa standard 12 lead resting ECG—2 groups of 6 leads with 5 seconds ofdata per lead group. This screen is also used to illustrate some of theinteractive features and display areas in accordance with the principlesof the present invention;

FIG. 5 is similar to FIG. 3 with the addition of the display feature toinclude the presentation of a median beat or other derived data. Thisfigure also illustrates the placement of annotation markers to identifywaveform features in accordance with the principles of the presentinvention;

FIG. 6 is similar to FIG. 4 with the addition of the display feature toinclude the presentation of a comparison ECG waveform in accordance withthe principles of the present invention;

FIG. 7 is an illustrative display screen that shows a 2× expandedwaveform to illustrate the equal expansion of the vertical andhorizontal aspects of the waveform display in accordance with theprinciples of the present invention;

FIG. 8 is an illustrative display screen that shows examples of thewaveform measurement display for the current marker, the current lead,and the overall summary measurements in accordance with the principlesof the present invention. This figure also shows a 4× expanded waveformdisplay in accordance with the principles of the present invention;

FIG. 9A is an illustrative display screen that shows a 8× expandedwaveform to illustrate the equal expansion of the vertical andhorizontal aspects of the waveform display in accordance with theprinciples of the present invention. This figure also illustrates theability to display a single lead for detailed analysis in accordancewith the principles of the present invention; FIG. 9B is an illustrativedisplay screen that shows an example of the waveform in FIG. 9Adisplayed at ½ scale (vertical scale is reduced to ½ while thehorizontal scale is maintained) in accordance with the principles of thepresent invention;

FIG. 10 is an illustrative display screen that shows an example textreport display and editing function with related data entry and optionselection features in accordance with the principles of the presentinvention.

FIG. 11 is an illustrative display screen that shows an example thatextends the editing features to include the menu selection of coded andtextual statements that will become part of the ECG report in accordancewith the principles of the present invention;

FIG. 12 is a conceptual block diagram of a system that may employ morethan one interactive display in accordance with the principles of theinvention. This diagram shows the potential interactions between systemelements and interactive displays in accordance with the principles ofthe present invention.

FIG. 13 is a conceptual detailed block diagram of a system that mayemploy additional analysis functions in support of the interactivedisplay in accordance with the principles of the invention. This diagramalso illustrates how the system may employ additional queuing and datacontrol functions in support of the interactive display in accordancewith the principles of the invention.

FIG. 14 is an illustrative display screen that shows an example of howderived data such as the time derivative of the time series waveformdata may be presented with the original data to assist in the analysisof the data. This figure also illustrates an alternative method for thedisplay of comparison time series waveform data may be displayed inaccordance with the principles of the current invention;

FIG. 15 is an illustrative display screen that shows an example of themenu that may be used to indicate acceptance of the edited report andenter instructions for subsequent processing in accordance with theprinciples of the present invention;

FIG. 16 is an illustrative display screen that shows an example of anexit screen that may be presented to a user when it has completed areview session; and

FIG. 17 is an illustrative display screen that shows that ancillarylines can be presented on a display with specific relationships.

DETAILED DESCRIPTION

FIG. 1 illustrates the system architecture for a computer system 100 onwhich the invention may be implemented. The exemplary computer system ofFIG. 1 is for descriptive purposes only. Although the description mayrefer to terms commonly used in describing particular computer systems,the description and concepts equally apply to other systems, includingsystems having architectures dissimilar to FIG. 1.

Computer system 100 includes a central processing unit (CPU) 105, whichmay be implemented with a conventional microprocessor, a random accessmemory (RAM) 110 for temporary storage of information, and a read onlymemory (ROM) 115 for permanent storage of information. A memorycontroller 120 is provided for controlling RAM 110.

A bus 130 interconnects the components of computer system 100. A buscontroller 125 is provided for controlling bus 130. An interruptcontroller 135 is used for receiving and processing various interruptsignals from the system components.

Mass storage may be provided by diskette 142, CD ROM 147, or hard drive152. Data and software may be exchanged with computer system 100 viaremovable media such as diskette 142 and CD ROM 147. Diskette 142 isinsertable into diskette drive 141 which is, in turn, connected to bus130 by a controller 140. Similarly, CD ROM 147 is insertable into CD ROMdrive 146 which is, in turn, connected to bus 130 by controller 145.Hard disc 152 is part of a fixed disc drive 151 which is connected tobus 130 by controller 150.

User input to computer system 100 may be provided by a number ofdevices. For example, a keyboard 156 and mouse 157 are connected to bus130 by controller 155. An audio transducer 196, which may act as both amicrophone and a speaker, is connected to bus 130 by audio controller197, as illustrated. It will be obvious to those reasonably skilled inthe art that other input devices, such as a pen and/or tabloid may beconnected to bus 130 and an appropriate controller and software, asrequired. DMA controller 160 is provided for performing direct memoryaccess to RAM 110. A visual display is generated by video controller 165which controls video display 170. Computer system 100 also includes acommunications adaptor 190 which allows the system to be interconnectedto a local area network (LAN) or a wide area network (WAN),schematically illustrated by bus 191 and network 195. An input interface199 operates in conjunction with an input device 193 to permit a user tosend information, whether command and control, data, or other types ofinformation, to the system 100. The input device and interface may beany of a number of common interface devices, such as a joystick, atouch-pad, a touch-screen, a speech-recognition device, or other knowninput device.

Operation of computer system 100 is generally controlled and coordinatedby operating system software. The operating system controls allocationof system resources and performs tasks such as processing scheduling,memory management, networking, and I/O services, among things. Inparticular, an operating system resident in system memory and running onCPU 105 coordinates the operation of the other elements of computersystem 100. The present invention may be implemented with any number ofcommercially available operating systems including Windows, OS/2, UNIXand DOS, etc. One or more applications may also run on the CPU 105. Ifthe operating system is a true multitasking operating system, multipleapplications may execute simultaneously.

As will be understood by those skilled in the art, Object-OrientedProgramming (OOP) techniques involve the definition, creation, use anddestruction of “objects”. These objects are software entities comprisingdata elements, or attributes, and methods, or functions, whichmanipulate the data elements. The attributes and related methods aretreated by the software as an entity and can be created, used anddeleted as if they were a single item. Together, the attributes andmethods enable objects to model virtually any real-world entity in termsof its characteristics, which can be represented by the data elements,and its behavior, which can be represented by its data manipulationfunctions. In this way, objects can model concrete things like peopleand computers, and they can also model abstract concepts like numbers orgeometrical designs.

Objects are defined by creating “classes” which are not objectsthemselves, but which act as templates that instruct the compiler how toconstruct the actual object. A class may, for example, specify thenumber and type of data variables and the steps involved in the methodswhich manipulate the data. When an object-oriented program is compiled,the class code is compiled into the program, but no objects exist.Therefore, none of the variables or data structures in the compiledprogram exist or have any memory allotted to them. An object is actuallycreated by the program at runtime by means of a special function calleda constructor which uses the corresponding class definition andadditional information, such as arguments provided during objectcreation, to construct the object. Likewise, objects are destroyed by aspecial function called a destructor. Objects may be used by using theirdata and invoking their functions. When an object is created at runtimememory is allotted and data structures are created.

The principle benefits of object-oriented programming techniques ariseout of three basic principles; encapsulation, polymorphism andinheritance. More specifically, objects can be designed to hide, orencapsulate, all, or a portion of, the internal data structure and theinternal functions. More particularly, during program design, a programdeveloper can define objects in which all or some of the attributes andall or some of the related functions are considered “private” or for useonly by the object itself. Other data or functions can be declared“public” or available for use by other programs. Access to the privatevariables by other programs can be controlled by defining publicfunctions for an object which access the object's private data. Thepublic functions form a controlled and consistent interface between theprivate data and the “outside” world. Any attempt to write program codewhich directly accesses the private variables causes the compiler togenerate an error during program compilation which error stops thecompilation process and prevents the program from being run.

Polymorphism is a concept which allows objects and functions which havethe same overall format, but which work with different data, to functiondifferently in order to produce consistent results. For example, anaddition function may be defined as variable A plus variable B (A+B) andthis same format can be used whether the A and B are numbers, charactersor dollars and cents. However, the actual program code which performsthe addition may differ widely depending on the type of variables thatcomprise A and B. Polymorphism allows three separate functiondefinitions to be written, one for each type of variable (numbers,characters and dollars). After the functions have been defined, aprogram can later refer to the addition function by its common format(A+B) and, at runtime, the program will determine which of the threefunctions is actually called by examining the variable types.Polymorphism allows similar functions which produce analogous results tobe “grouped” in the program source code to produce a more logical andclear program flow.

The third principle which underlies object-oriented programming isinheritance, which allows program developers to easily reusepre-existing programs and to avoid creating software from scratch. Theprinciple of inheritance allows a software developer to declare classes(and the objects which are later created from them) as related.Specifically, classes may be designated as subclasses of other baseclasses. A subclass “inherits” and has access to all of the publicfunctions of its base classes just as if these function appeared in thesubclass. Alternatively, a subclass can override some or all of itsinherited functions or may modify some or all of its inherited functionsmerely by defining a new function with the same form (overriding ormodification does not alter the function in the base class, but merelymodifies the use of the function in the subclass). The creation of a newsubclass which has some of the functionality (with selectivemodification) of another class allows software developers to easilycustomize existing code to meet their particular needs.

FIG. 2 illustrates conceptually the main components of an interactivedisplay system 200 in accordance with the present invention. A userinput device 202 may take the form of a known user input device anddevice interface, such as keyboard and mouse (with correspondingcontrollers), a joystick, touch pad, touch screen, voice input device,etc. in combination with controllers that may be embodied as variousinstantiations of object classes. The new interactive display engine 204may include various ones of the hardware components described in thediscussion related to FIG. 1. The interactive display engine 204 isconfigured to display time-series data, such as electrocardiogram (ECG)data, in graphical form, on the display 206. In one aspect of aninteractive display in accordance with the principles of the presentinvention, electrocardiograms are displayed using a format that is thesame as, or substantially similar to, standard paper recording formats.For example, data from a standard ECG recording may be displayed astraces overlaid on a millimeter grid reference background. By displayingECG data in much the same format as that of a conventional ECG paperprintout, the system capitalizes on the pattern recognition skillsdeveloped by cardiologists through years of training and experience.Moreover, the interactive display system maintains the aspect ratio,thereby preserving the pattern-recognition advantages, during electronicmagnification (zoom) operations. By preserving the aspect ratio in thismanner, doubling, for example, both the horizontal and vertical scalesfor both the ECG waveform and the reference grid for a 2× magnified viewof an ECG, a cardiologist may make precise, highly-refined ECGmeasurements, even while viewing an undistorted representation of theECG.

The interactive display engine 204 accepts input from the user inputdevice 202. In response to the user input, the system produces outputfor the display 206 and, depending upon the user input, may record theuser input. For example, if the user selects a marker and positions iton the display, the display engine 204 repositions the marker on thedisplay. If the user also selects the marker position, by activating an“enter” key on a keyboard or “double-clicking” a mouse-button, forexample the display engine 204 also records the position selected by theuser. The engine 204 may also correlate, by computation for example, theunderlying data values associated with the selected screen position,then store the screen position, underlying data, and other values. Anillustrative object-oriented embodiment of the interactive displayengine includes object classes that: read data into onscreenmeasurements, provide file interfaces, provide standard interfaces todifferent ECG source types, provide standard interfaces to waveforms,provide forms that allow the selection of interpretive codes forinclusion in a code and comment segment of the report form, to pass amodified ECG to another stage in a review process, to provide displayand calculation options, to display report file text and measurementsfrom on-screen analysis and to add assessment codes and/or comments, andto select an ECG file from disk storage.

A time series medical data source 208 may take the form of digitizedECGs stored in computer files, data obtained locally from a subject, orECGs obtained through a telecommunications link 210 from one or moreremote patient locations 212, for example. ECGs obtained through atelecommunications link 210 may be stored locally and processed inprioritized order, as will be described in the discussion related toFIG. 12, for example. In an illustrative embodiment, the interactivedisplay engine 204 presents electrocardiogram data in a manner that issubstantially the same as one of the standard paper recording formats.For example, data from a 12 lead resting ECG may be displayed aswaveform traces overlaid on a millimeter grid background. By displayingECG data in much the same format as that of a conventional ECG paperprintout, the system capitalizes on the pattern recognition skillsdeveloped by cardiologists through years of training and experience.When the interactive display is used to make on-screen measurements, thetraces may be “magnified” in a number of different formats, includingone in which the aspect ratio of the displayed ECG is maintained. Thispermits the interactive display to preserve the pattern-recognitionadvantages of the display, while providing for more accurate placementof markers and a concomitant improvement in the precision of on-screenmeasurements. By preserving the aspect ratio in this manner, doubling,for example, both the horizontal and vertical ECG scales for a 2×magnified view of an ECG, a cardiologist may make precise,highly-refined ECG measurements, even while viewing an undistortedrepresentation of the ECG.

The time-series data source 208 may be obtained directly from anelectrocardiogram machine that provides digital output. If theelectrocardiogram machine provides only analog output, the analogsignal(s) may be converted to digital form for processing by aninteractive display in accordance with the principles of the presentinvention. Whether the analog ECG signals are converted to digital formby the ECG machine or in post-processing, a user may process thecorresponding digital data on the interactive display directly from theECG machine or from stored ECG data. Additionally, one or more users mayuse an interactive display in accordance with the principles of thepresent invention to process ECG data that is obtained at one or morepatient sites and transmitted, via a telecommunications network, forexample, to an ECG analysis center. Digitized ECG data may betransmitted to an analysis center and stored for future processing orplaced in a queue for immediate processing. Alarms of varying degrees ofurgency may be activated by an interactive display in accordance withthe principles of the present invention in response to ECG dataanalyses.

In various centralized embodiments of time series medical data analysissystems in accordance with the principles of the present invention,analysis and/or markup of time series medical data may be performed at acentralized location, with data supplied to the centralized locationthrough a telecommunications network 210, for example. The displayengine 204 may be responsive to signals from the user input 202 bymodifying the size, shape, position, or other aspects of the marker.Such modifications to the marker are provided as a visual feedbackmechanism for a user. In addition to the marker modification, thecontroller may operate on data, or provide an indication to anothercontroller that the data should be operated upon, in a predeterminedmanner corresponding to the manipulation of the marker. For example, atime interval may be marked off by the manipulation of one or moremarkers, and the engine 204 may respond, not only by positioning themarkers according to user input, but also by computing the time and/orother values (e.g., average signal level over the interval).Additionally, modifications to the position of the marker may bereflected by modifications to information displayed, with, for example,the coordinates of one or more markers displayed and updated “on thefly” as a marker is repositioned on the display. The coordinates mayrepresent a multi-dimensional space in which dimensions are devoted tosignal level, time, event number, or other variables, for example.Markers may be combined with other interactive display devices andtechniques such as pull-down or popup menus, buttons, or sliders, forexample.

In an illustrative embodiment, the interactive display presentselectrocardiogram data in a manner that emulates conventionalstrip-chart recorders. That is, the ECG data are displayed as waveformtraces overlaid on a millimeter grid background. In a display mode thatmay be used as a default, each grid division, or cell, represents 40milliseconds along the abscissa and 0.1 millivolt along the ordinate.Multi-grid divisions may be “set off” by employing heavier grid linesevery fifth division, for example, to form 200 millisecond by 0.5millivolt “super-cells.” A 1280×1024 pixel display that employs an1220×690 pixel area for the display of waveforms may employ one pixelfor every eight milliseconds (five pixels per millimeter) in displayinga standard 1 mm×1 mm, 40 ms×0.1 mV grid. Various “zoom” schemes may beemployed to increase or decrease the resolution of the display byincreasing or decreasing the number of pixels dedicated to eachmillisecond and/or millivolt. A user may select one or more features ofinterest by manipulating one or more markers. Each marker may be aconventional marker, such as a cursor such as is used in word-processingapplications, for example. Alternately, a marker may be a vertical linethat intersects the waveform and has an associated alphanumericcharacter, such as a “P” (corresponding to the beginning of atrialdepolarization) to identify the meaning of the marked point. The markermay also be iconic, such as a graphic corresponding to a frontal planeQRS axis, for example. Intervals, such as PR, QRS, QT, and RR intervals,may be measured on-screen using one or more markers. That is, forexample, a user may employ a single marker to indicate a point for whichmeasurements, such as voltage and time, are desired. The user may use asingle marker to set the beginning of an interval for which measurementsare desired, then place another marker at the end of the desiredinterval, or a plurality of markers may be employed to denote datavalues or data intervals of interest.

Multiple markers may also be employed, with one assigned to mark thebeginning of an interval and one assigned to mark the end of an intervalof interest, for example. The interactive display system also providesfor “automatic” marking. For example, in one mode of operation a usermay mark a “Q” feature on a trace of interest in response to which thesystem completes the markings for the P, J, and T points, or features,and computes the QRS interval and other measurements associated with themarked points and associated medically significant artifact.Measurements may be displayed in close proximity to the measurementmarks, overlaid on the strip chart background, and/or may be displayedin one or more separate “reporting areas” on the display. The displayedmeasurements may be updated continuously, so that, as a marker is movedacross the strip chart background, the measurement display continuouslychanges to reflect the updated position of the marker. Alternatively,the displayed measurements may be updated upon finalization of a markerposition. The finalization may be effected through use of an “enter”keystroke, for example.

The time-series data may be obtained directly from an electrocardiogrammachine that provides digital output. If the electrocardiogram machineprovides only analog output, the analog signal(s) may be converted todigital form for processing by an interactive display in accordance withthe principles of the present invention. Whether the analog ECG signalsare converted to digital form by the ECG machine or in post-processing,a user may process the corresponding digital data on the interactivedisplay directly from the ECG machine or from stored ECG data.

The screen shot of FIG. 3 is illustrative of a display output inaccordance with the principles of the present invention. A user mayselect one or more features of interest by manipulating one or moremarkers. Each marker may be a conventional marker, such as a cursor suchas is used in word-processing applications, for example. Alternately, amarker may be a vertical line that intersects the waveform and has anassociated alphanumeric character, such as a “P” (corresponding to thebeginning of atrial depolarization) to identify the meaning of themarked point. The marker may also be iconic, such as a graphiccorresponding to a frontal plane QRS axis, for example. Intervals, suchas PR, QRS, QT, and RR intervals, may be measured on-screen using one ormore markers. That is, for example, a user may employ a single marker toindicate a point for which measurements, such as voltage and time, aredesired. The user may use a single marker to set the beginning of aninterval for which measurements are desired, then place another markerat the end of the desired interval, or a plurality of markers may beemployed to denote data values or data intervals of interest.

Multiple markers may also be employed, with one assigned to mark thebeginning of an interval and one assigned to mark the end of an intervalof interest, for example. The interactive display system also providesfor “automatic” marking. For example, in one mode of operation a usermay mark a “Q” feature on a trace of interest in response to which thesystem completes the markings for the P, J, and T markers and computesthe measurements associated with the beat, that is, the PR interval, theQRS duration, and the QT interval. Once a typical set of markers havebeen placed, the system may sequence through all beats and place markersfor these points. The system may operate through other automaticsequences related to the selection of particular measurement points,such as the location and measurement of a point to measure ST elevationor depression. An operator may define a set of marks as an auto-sequenceto allow the system to replicate measurements based on the samplemarkers placed during the definition. Sequences of marks may beautomatically placed either through operator initiation of the operationor based on specific requirements for sets of ECGs where annotationmarks will be automatically placed for operator verification. Suchfeatures as automatically marking the peaks of waveform features as wellas the onset and offset of detailed features are supported by theprinciples of the present invention.

Measurements may be displayed in close proximity to the measurementmarks, overlaid on the strip chart background, and/or may be displayedin one or more separate “reporting”_ areas on the display. For example,in the illustrative screen shot of FIG. 3, an information button 300 maybe used to alert an operator in some way. For example, in thisillustrative screen shot, the button 300 features a white button on ared background to indicate an emergency read condition. By activatingthe button (e.g., by “clicking on” the button) the operator may obtainmore information related to the ECG data set, related, for example, tothe nature of the emergency. Various indicators, such as flashing, colorchanges, the use of specific colors, different levels of transparency,and other display techniques may be used to alert an operator to variousconditions related to the button 300 or other features displayed on aninteractive display in accordance with the principles of the presentinvention. Other conditions may be signaled by operation of the button300. For example, a blue background with a white “i” substituted for theX, may be used to indicate that further information is available to anoperator and that it may be obtained by clicking on the button 300.Protocol information may be obtained through activation of the protocolbutton 302. In this illustrative example, the start time, resolution ofthe display, and the number of leads for which data are displayed arerespectively displayed in windows 304, 306, and 308. The windows mayinclude up/down arrows, such as arrows 310 to allow an operator toselect different resolutions or number of leads displayed, for example.

The display may also include an indication of the number of jobs waitingto be serviced, as indicated by the window 312. The start time is theinitial offset of the starting point on the screen with reference to thebeginning point of data collection. The “resolution” feature included inthis illustrative display indicates the relationship of the waveformdisplay to the physical display. The 8 ms resolution means 8 ms/pixel.This is the standard starting display resolution in this illustrativeembodiment. By decreasing the resolution, the magnification of thedisplayed waveform is increased. Decreasing the resolution to 4 ms woulddouble the size of the displayed wavefonn The display of the backgroundmillimeter grid is also based on the resolution setting. If the numberof pixels on the display between 1 millimeter grid marks would cause thegrid to make the waveform difficult to read, the background grid isreduced to 5 millimeter increments. An option is also provided totemporarily eliminate the background grid if desired. The invention isnot limited to the concept of using the smallest viewable feature of thedisplay as the maximum resolution. The “Leads” option 308 permits anoperator to increase or decrease the number of leads that are displayedvertically on the screen. The system will attempt to optimize the numberof leads being displayed based on average waveforms or on the amplitudeof the waveforms being presented. This option allows viewing a singlelead regardless of the magnification. The “Jobs in Queue” field 312indicates to an operator the number of separate “tracings”, e.g., ECGs,waiting to be processed by the operator on the system. In anillustrative embodiment that employs a display that utilizes 1220 ×690pixels of display area for the display of traces, when 8-millisecondresolution is selected, the screen is able to display 10 seconds ofdata. At 4-millisecond resolution, 5 seconds of data can be displayed.The resolution indicates the amount of time represented by each screenpixel as well as the change in marker location when the left or rightarrow keys are pressed.

Summary measurements may be displayed in windows labeled HR, Max P-R,Max QRS, Avg R-R, Max Q-T, and QTc, for example. The type and sequenceof marker(s) to be placed on a display may be selected from the MarkerPlacement display menu. The options in this menu include areas labeledPQJT Points, R-R intervals, Q-T Intervals, P Intervals, QRS Duration, Qauto PJT, and Event Annotation, for example. Data associated with leadcontaining the “active marker”, that is, the marker currently beingmanipulated, may be displayed in the windows labeled Lead, PR, QRS, andQT. Additional windows, such as windows 314, 316 and 318 may be used todisplay such information as the values for all measurements based on thecurrent active marker (indicated in box 314), and the specifiedcalculated values for all measurements in the currently active lead.Items 316 and 318 are display boxes that provide information about datathat has been provided with the current ECG. This information includespreviously calculated measurement values that have been provided andwhen the last review of the ECG was done and by whom.

In an illustrative embodiment, the waveform display area includes 5-mmgrid lines 320 for reference. Each grid division represents 200 ms inthe time dimension and 0.5 mV in the voltage dimension. The perspectiveof the waveform may be maintained during a “zoom” operation bymagnifying the horizontal and vertical dimensions of the waveform by thesame amount. In an illustrative embodiment, the interactive displaysupports zoom operations that allow an operator to place markers withgreater precision than a full-screen twelve-lead display might otherwisepermit. The number of pixels between grid lines varies in the process ofzooming and 1 millimeter grid lines may be added during a “zoom in” inorder to provide the standard recognizable grid background for visualreference. Conversely, 1 millimeter grid lines may be deleted during a“zoom out” operation in order to avoid cluttering the display. Fivemillimeter grid lines will always be present (unless the grid linedisplay option is turned off). In an illustrative embodiment, afive-pixel threshold is employed whereby 1 mm grid lines are added tothe display when five or more pixels are required to display a distanceequal to one mm. To provide additional context, every fifth grid linemay be distinguished from the other grid lines by displaying them wider,darker, or in a different color, for example.

The display may provide for a variety of display modes, depending on theavailable data and the preferences on the operator. For example, acomplete 12 lead resting ECG may be displayed as 12 leads by 10 seconds.It may also be displayed as 2 five second groups of 6 leads each.Depending upon the display output device, display magnification, alsoreferred to herein as zooming, may cause a portion of the data to exceedthe display area available at any one time. The display may providehorizontal and/or vertical scrolling to provide access to such displayinformation. Additionally, as the magnification of a waveform isincreased, the number of leads that can be displayed vertically withoutoverlap may be reduced. The number of leads to be displayed can eitherbe set automatically by the program to optimize the number of lead to bedisplayed or it can be manually selected to increase or decrease thenumber of leads being displayed. An operator may select a subset of the12 leads to be displayed.

The waveform's trace should be wide enough and heavy enough to make itreadily viewable at a nominal viewing distance, yet should not be sowide and heavy as to obscure the waveform features. In an illustrativeembodiment the waveform is plotted on the screen with the minimal linewidth that makes the waveform readily viewable with the reference grid.However, the display includes a facility that allows an operator toadjust the waveform trace's line-width to conform to the operator'svisual requirements.

The screen shot of FIG. 4 illustrates an alternate 12 lead waveformdisplay screen view in accordance with the principles of the presentinvention. A vertical line 400 separates the display into two sets ofsix leads each. In this illustrative embodiment, the I, II, III, aVR,aVL,and aVF lead data are displayed on the left-hand side of thevertical line 400 and the V1, V2, V3, V4, V5, and V6 data are displayedon the right-hand side of the vertical line 400. The right- andleft-hand sides of the display may be sequential depending on theavailable data. That is, for example the data plotted for the V1 leadstarts at the time the data plotted for the I lead ends. This displayformat may also be used to display waveforms that are not acquiredsimultaneously. A number of markers have been placed on the screen. Inthis illustrative embodiment, the markers are vertical lines used todesignate the exact position chosen by an operator. Additionally, themarkers are multi-part, in that they are accompanied by alphanumericlabels (e.g., P,Q J, etc.), corresponding to familiar labels given toECG features.

The screen shot of FIG. 5 illustrates a split-screen display inaccordance with the principles of the present invention in which a“pane” 500, delineated by vertical line 502, may be opened to displaymedian heart beats or other derived waveform data. The median beats paneis automatically sized to display the entire beat, and so does notscroll with the rhythm waveforms. In an illustrative embodiment aninteractive display in accordance with the principles of the presentinvention provides one or more markers, such as markers 504 and 506, foruse by an operator in an on-screen measurement process involving one ormore displayed time-series waveforms. Markers 504 and 506 includealphanumerical components, indicating, in this instance, the beginningand end of an interval measurement. The markers are used to identifyon-screen measurement points and are plotted vertically and identifiedwith a label of the point or event that is being identified: the “begin”and “end” labels in this example. In an illustrative embodiment, when amarker is associated with a single lead, the marker will be limited tothe plot area occupied by that lead. When global measurement marks arebeing used, the mark will extend from the top to the bottom of the plotscreen area. The width of the markers may be selectable as an option.Initial placement of the markers may be with the use of a light pen ormouse pointer and a click. Once a marker has been placed, it becomes the“active” marker. The active marker can be moved left or right, with theresolution of one pixel at a time. When moving a marker, the timeincrement (the time represented by each pixel) will vary depending onthe selected resolution. The active marker is the one most recentlyadded, or one that has been selected. The active marker may be selectedusing a keyboard, mouse pointer, or light pen, for example. Theinteractive display may provide visual feedback to a user by, displayingthe active marker with a different color than other displayed markers,by flashing the marker, or using other display techniques. Aninteractive display in accordance with the principles of the presentinvention may also allow a user to select the color of markers and toselect the means of highlighting the active marker.

As previously noted, in accordance with the principles of the presentinvention, a twelve-lead resting ECG may be displayed in the formatnormally presented on the printed page. Lead I and lead V1 will beplotted on the same line with lead V1 starting at exactly 5 secondsafter the start of lead I. The other sets of leads will be plotted inthe same manner: II and V2, III and V3, aVR and V4, aVL and V5, and aVFand V6. The waveforms for the 60 seconds of rhythm data will bedisplayed as a single lead continuous string of waveform data. Thisformat supports making sequences of measurements across the entire setof data. The single or multi-lead waveforms for 60, 120 seconds, orlonger sets of rhythm data may be displayed, for example.

As illustrated by the screen shot of FIG. 6, comparison ECGs may also bedisplayed, using toolbar activation, file menu selection, or other userinterface techniques. In an illustrative embodiment, the interactivedisplay permits an operator to view a comparison ECG on its own, or, asillustrated, in a side-by-side configuration of two panes 600 and 602.Report text associated with the comparison ECG may also be displayedunder operator control, by selection from a popup menu, for example. Aplurality of comparison ECGs may be viewed, for example by selecting“next comparison” from a popup menu.

The screen shot of FIG. 7 will be used to illustrate in more detail theplacement of markers on an ECG display in accordance with the principlesof the present invention. To place a marker on the waveform, the usermay select a marker placement sequence by clicking on a marker placementcontrol: one of the chad-like selection mechanisms labeled PQJT Points,R-R Intervals, etc. The interactive display will start with the firstmark in the selected sequence and will provide subsequent markers in theproper sequence. An interactive display in accordance with the presentinvention also permits an operator to annotate events, for example, bymarking the beginning and end of a section of a Lead wave and attachingdescriptive text to that segment. Such markers may be used to delimitand describe a section of the Lead wave without creating a measuredinterval. In particular, they may be used to delimit and describe asection of the Lead wave, such as an ST depression, or abnormal U-wave.In an illustrative embodiment, when one of these markers is selected,its descriptive text is displayed in the Status Bar at the bottom of theform. In an illustrative embodiment, a sequence of markers may be placedby an operator clicking and hold the left mouse button to initiatemarker placement. The interactive display may then display a dottedvertical line to indicate where the marker will be placed. The user maythen move the mouse to the desired location and release the button toplace the marker. The selected marker placement sequence (Q Auto PJT inthis example, would first place the Q marker), will determine the nextmarker type to be placed. The label for the next marker is displayed inthe marker placement control.

In an illustrative embodiment, after placement, the active markerposition can be adjusted using arrow keys or other user input devices.To select the active marker using an arrow key implementation, the usermay press shift and the left or right arrow key until the desired markeris highlighted, for example. To move the active marker, the user couldthen press the left or right arrow keys or use an alternate graphicinput device to indicate the marker should be moved. In thisillustrative embodiment, each time the arrow key is pressed, the markermoves by the amount indicated in the current resolution and intervalcalculations are instantly updated as a marker is moved. An interactivedisplay in accordance with the principles of the present invention mayalso provide keystroke functions to select the first and last markers asthe active marker and to shift the area of the waveform being displayed.The Home key or some other indicator may be used to center the activemarker in the viewable screen area.

The screen shot of FIG. 8 illustrates the display of active marker datain accordance with the principles of the present invention. Whenmatching sets of markers have been placed, intervals are calculated fromtheir positions and displayed in several boxes at the top of thewaveform. In the active marker area 802, the intervals calculated usinga method for the current lead (the lead containing the active marker)are displayed. The method used for calculation of the measured valuesmay be selected by the operator or may be set by information whichaccompanies the ECG waveform data. Directly below that (800) aredisplayed all intervals with which the active marker is associated;moving the active marker will typically impact both of these sets ofintervals. In the summary measurements area 804, the overallmeasurements are displayed; these measurements are derived from markerson all leads. The calculated values may represent, for example, eitherthe maximum lead average for which multiple intervals are marked, or theoverall maximum of each measurement, depending on the current setting ofthe summary measurement calculation option. Placing the mouse pointerover the Max P-R, Max QRS, or Max Q-T values will present a ToolTipdisplaying the Lead on which the maximum value was found. In addition,double-clicking on the Max P-R, Max QRS, or Max Q-T values willhighlight and make active the Markers representing the maximum measuredinterval of that measurement.

The screen shots of FIG. 9A and FIG. 9B illustrate the use of ahalf-scale display in accordance with the principles of the presentinvention. This optional view may be used, for example, for an ECG thatexhibits very high amplitudes, resulting in the waveforms being drawnoutside the top (902) or bottom of the display area or overlapping eachother. Note that the scale option may be exercised independently of theresolution option. When the half-scale display option is selected, thisstatus is indicated with an on-screen display 910 and with changing thecolor or other attribute of the plotted waveform, for example. Variousfilter options are accessible, in this illustrative embodiment throughinteraction with a toolbar. For example, as indicated by the tool boxeslabeled “No Filter”, “50 Hz”, and “60 Hz”, a operator may select 50 Hz,60 Hz, or no filtering of the ECG data. These filter options may be usedto assist in the analysis of noisy data without affecting the originaldata, for example.

The screen shot of FIG. 10 depicts a report form format for aninteractive display in accordance with the principles of the presentinvention. The report form includes a window 1000 that displays reportinformation. The Original Report Text area can be used to display theresults of a computer analysis of the ECG or the results of a previousoperator review, for example. The Measurement Intervals area 1014 candisplay the results of the calculations used to determine measurementvalues from the markers and that will be included in the completedreport. The Codes and Comments area, field 1008, may be used to enterinterpretive and diagnostic codes and statements and additional commentsthat are to be part of the completed report. The values in this area maybe preset by computer interpretation, prior review, or other data thatmay accompany the ECG, such as the customer ID number, for example.

In this illustrative embodiment, the interactive display includes afield 1002 of comparison statements selection options that may be usedto conveniently note trends in ECG data for a particular patient basedon a previous ECG. An operator may also employ the severity field 1004to enter information related to the overall assessment of the ECG. Asession signature field 1006 may be used to display the current date andtime and the identification of the current operator. A separate entry isrequired by the operator to be the electronic signature for the set ofentries.

In this illustrative embodiment, the display includes a Save function1012 to indicate that the report editing process is complete.

In this illustrative embodiment, the display includes an InterpretiveCodes function 1016 to allow the selection of interpretive codes from amenu. An illustrative embodiment of the code menu selection screen isshown in FIG. 11.

As illustrated by the screen shot of FIG. 11, an interactive display inaccordance with the principles of the present invention provides a toolfor an operator to select among a variety of interpretive codes. Thetypes of codes are indicated by folder tabs 1101, marked “overallassessment”, “comparison”, “rhythm”, A-V “conduction”, “ST segment”,etc. Each tab contains a set of code numbers and text related to thelabel on the tab. These code values, when selected, will be placed inthe codes and comments section, field 1008, for inclusion in the report.The complete list of selected codes is displayed for reference in window1003. Provision is made for changes to measurements or observations thatare not determined by markers, the QRS axis for example 1104. Placingthe selected codes in the Codes and Comments are 1008 is accomplished byclicking on the OK button 1105, for example.

Additionally, one or more users may use an interactive display inaccordance with the principles of the present invention to process ECGdata that is obtained at one or more patient sites and transmitted, viaa telecommunications network, for example, to an ECG analysis center.The block flow diagram of FIG. 12 depicts the flow of processing ECGdata in a central processing center in accordance with the principles ofthe present invention. Processes illustrated through the use of flowcharts or block flow diagrams may not be strictly linear processes andalternative flows may be implemented within the scope of the invention.The specific configuration of logic and/or instructions utilized toachieve a particular function, as well as other modifications to theinventive concept are contemplated within the scope of this invention.

In a time-series data processing center in accordance with theprinciples of the present invention, digitized ECG data 1200 may betransmitted to an analysis center and stored for future processing orplaced in a queue for immediate processing. Alarms of varying degrees ofurgency may be activated by an interactive display in accordance withthe principles of the present invention in response to ECG dataanalyses. In addition to ECG data, the data 1200 may include the serialnumber of a unit, such as an ECG machine, revision code of program, datacontrol card number, type of test being transmitted (12 lead, 60 second,120 second rhythm and related lead), whether this particular ECG set hasbeen transmitted before, whether battery voltage is low, for example.

In this illustrative embodiment, data is received and temporarily storedin a data control and queue manager 1202. After the file is received bythe data control and queue manager (which could be an instantiation ofan object class, for example) 1202 the data file is sent in step 1203 todata dependent analysis programs 1204. The computer analysis programdetermines annotation points and creates tables of measurements fromthese points. The analysis program may also infer, from thesemeasurements and other data provided with the ECG, an assessment andinterpretation of the ECG. The results of the analysis may be recordedin a report format similar to the intended output of the invention.Different analysis programs may be necessary to perform different setsof measurements and interpretations based on the type of ECG data beingprovided and the reason for processing the ECG. Some or all of theresults of the computer analysis may be made available to the systemincorporating the invention to provide initial marker points oradditional information about the ECG waveform.

In step 1205 results from these programs are shipped back to the datacontrol and queue manager 1202. From there, the results (which includemeasurements and waveform information) are sent to a viewing station1206 in step 1207. At this point, an operator, User 1, may enter ormodify markers used to determine interval measurements, for example. Ifchanges are made, the updated data may be returned to the data dependentanalysis programs in step 1215 and the files are sent back to the queuemanager in step 1209.

The “marked up” waveform data, with markers and intervals, may be sentto a viewing station, 1208, in step 1211. Although the viewing station1208 may be the same viewing station, in an illustrative embodiment theviewing station is a separate viewing station for use by a secondoperator, User2. User 2 views the data and adjusts the markers as hefeels necessary. User 2 may make, for example, changes and additionsthat he feels are necessary, along with his comments and observationsabout the waveform. The end results, including all annotations, aretransferred in report and results files in step 1219 to an automaticimporter. In an illustrative embodiment, a plurality of viewing stations1206 may receive time series data, such as ECG data, with the datadistributed by the data control and queue manager 1202 using call-centerrouting. In such an embodiment, incoming calls (e.g., ECG data) may bedirected to the next available user. However, calls need not beprocessed in the order in which they are received at the center. Forexample, emergency calls may be given the highest priority, with othercalls being assigned descending priorities depending on their processingrequirements.

As an illustrative example, the process whereby an ECG or other timeseries data may be reviewed by a sequence of 2 users at two interactiveviewing stations in accordance with the principles of the presentinvention is shown in FIG. 12. The processing sequence may be controlledby 1202 Data Control and Queue Management based on the usercapabilities, ECG specific processing requirements, or generalprocessing requirements, for example. As an illustrative example,additional user review steps may be added to the sequence if indicatedby processing requirements.

The block flow diagram of FIG. 13 illustrates in greater detail theprocess whereby a user may review, change and approve a processed ECGfile in accordance with the principles of the present invention. Data,report and results are stored in storage 1300 and made available to atechnician's workstation 1206 through the data control and queue manager1202. For the user's review, the queue manager 1202 sends current andcomparison ECGs, in the form of report and results files to interactivedisplay 1206 in step 1301. Analysis and measurement programs, whichreside in this illustrative embodiment on the interactive display inaccordance with the principles of the present invention, include a 12—lead analysis engine 1302 to provide full detailed measurement andanalysis of the ECG waveform or to provide a re-assessment of thewaveform based on changes made by the operator, a rhythm analysis engine1304 to provide a test specific set of measurements and assessments forsingle or multi-lead rhythm tests, and a special measurement engine 1306to provide non-standard measurements and assessments of 12 lead restingECGs, rhythm strips or other time series data that may be collected andpresented. In accordance with this illustrative embodiment, a user canaccept marked measurements, reject marked measurements (by moving one ormore markers), reprocess an interpretation if measurement changes aremade, or select alternative analysis/measurement processing.

The Current and Comparison Data Storage (shown in 1308) includes ECG jobdata, patient demographic data, customer specific processingrequirements, account information, such as a patient site and sponsordata, complete waveform display data, and the complete report text. Thereport file and associated measurement and database files retain theoriginal computer measurements and interpretation and the history of allcompleted reviews and processing, including the latest cardiologistrevisions. The results file and associated files and databases containmarker and measurement information entered or calculated by the systemto allow annotation points and measurements to be displayed by theinvention. This includes: complete measurement matrix values from theanalysis program, such as detailed measurement values for features ineach lead, for example; sample point locations and values for each ofthe following items for each of the 12 leads: such as P onset, Pamplitude, P duration, Q onset, Q amplitude, Q duration, R onset, Ramplitude, R duration, S onset, S amplitude, S duration, J point, Jlevel, J+80 ms level, T onset, T amplitude, and T duration, for example.Multiple sets of sample location points will be stored when specialmeasurements have been made. Each time a change is made to the contentsof the results file, all old results are retained and the new resultsare appended with the appropriate identifiers, including reviewingoperator, date, and time. In an illustrative embodiment, each time asession is completed, the system requires the entry of a password toverify the identity of the operator.

As illustrated in FIGS. 10 and 11, the current invention supports theentry of codes, text and comments related to the interpretation of thetime series data being presented and marked for measurement. In thecurrent invention, these entries may be checked for validity againsteach other and against the measurements determined by the markers aspart of a Quality Assurance process. The validity checking may result innot allowing certain combinations of contradictory entries or it mayallow entries to remain with confirmation that unusual values orcombinations of entries are correct. The Code Validation tab 1018 isused to display the results of this analysis for correction orconfirmation.

A user may select among various views and magnifications of ECG data, asillustrated by the screen shots of FIGS. 3, 4, 7, 8, 9A, and 9B. FIGS. 3illustrates a 12 lead, standard height standard width, with 12 leads ata time. FIG. 4 illustrates a 12 lead ECG in an alternate format showing2-6 lead by 5 second lead groups as previously described. These displaysmay be used, for example, to determine an overall evaluation of thedata. FIG. 7 illustrates a double-height double-width display of thesame ECG displayed in FIG. 4.This display may be used to make detailedmeasurements. FIGS. 8, 9A, and 9B illustrate increasing magnification ofthe waveform data to provide increased marker placement precision. FIGS.9A and 9B illustrate the option to display only a single lead of data.FIG. 9B illustrates the use of the ½ scale display option that may beused for large amplitude waveforms.

FIG. 14 illustrates an additional method of displaying derived orcalculated data in conjunction with the original time series data. Inthis figure, the first time derivative of the time series data 1402 ispresented just below the original data 1401. Activation of this displayoption may be with the selection of a check box 1400, for example. Thepresentation of this data may be used to assist in the analysis of theoriginal data by helping to determine where the maximum rate of changeof the data occurs, for example. This same figure also illustrates howthe time series data for a comparison waveform may be presented in closealignment with the original data for comparison as an alternative to theside by side comparison display illustrated in FIG. 6, for example.

FIG. 15 illustrates an additional method of the current invention fordisplaying the original time series data by plotting 2 or more sets ofthe time series data against each other rather than against time. As anillustrative example, the ECG lead values of lead I and lead aVF areplotted against each other 1500 to represent the frontal plane QRS axisloop. The plot results may be indicated with a label 1501 and activatedby a selection from a drop down menu, for example. The resultant twodimensional vector plot of heart activity within the body assists acardiologist in the analysis of a given ECG. FIG. 16 is an illustrativeexample of an exit screen that may be presented to a user when they havecompleted a review session. The process options 1600 allow the user toselect the next step or allow processing to continue as required. Theconfirmation password 1602 requires the confirmation by the user ofcompletion by entering their password.

As illustrated in FIG. 17, in response to user input or as a result ofcomputer analysis, ancillary lines may be drawn on the display withspecific relationships to the time series waveform to assist in theanalysis of and marker placement on the time series data. Theseancillary lines may take the form of placing an isoelectric line 1700with relation to the time series data or the placement of a tangent line1701 at a particular point on the data, for example.

In an illustrative embodiment a user initiates the operation of aninteractive time-series data display in accordance with the principlesof the present invention by a login process which may include securityprocesses that permit different levels of access to different users.After logging in, the system provides access to time-series data formeasurement. The time-series data, such as ECG data, may be stored andaccessed though a database manager, for example, or the data may beprovided to the interactive display “real time” by connection to an ECGmachine. In a central processing embodiment, data is transmitted fromone or more ECG machines to a central location for processing by one ormore interactive display systems in accordance with the principles ofthe present invention.

An interactive display system in accordance with the principles of thepresent invention provides an on-screen measurement function that allowsan operator, such as a cardiologist, to select one or more displayedpoints for the measurement of time-series data. The system captures, notonly the coordinates of the selected points, but corresponding waveformcoordinates, with a translation of the coordinates into “real world”values, in microvolts and milliseconds, for example. The system alsologs a record of all selections and edits and identifies the partiesresponsible for the selections. In an illustrative embodiment, theinteractive display system provides a report file and a measurementmatrix file related to such measurements. The report file, measurementfile, and associated transaction oriented data files may be individualfiles or may be configured as a set of database tables to contain allthe required data related to the analysis and assessment of the originaltime series data.

An interactive display system in accordance with the principles of thepresent invention may be used in the measurement, annotation, andanalysis of time-series data, such as ECG data. Various display formatsmay be employed, including those that emulate traditional strip-charthard-copy reports. The use of such traditional formats builds upon theexisting knowledge-base of those in the field, such as technicians andcardiologists, who have extensive training in the measurement, modeling,and analysis of such time-series records. The system may display datarelated to one or more channels, including time-series data from astandard twelve-lead ECG machine. Additional display formats, such asfrequency domain representations and other derived or processed data,may also be employed and displayed.

In an ECG measurement embodiment, the interactive display may provide avariety of display modes, including twelve-lead resting, 60-secondrhythm, 120-second, and longer rhythm displays. In a twelve-lead restingECG, the leads may be plotted in a variety of display arrangements tooptimize the information content of the display for the operator. Sucharrangements may include display of simultaneously acquired data so thatthe time relationship of the different waveforms is maintained foranalysis, as well as the ability to display data that is notsimultaneously acquired either in its time sequence acquired format orin a more compact and understandable printed report format. Examples ofthese types of display have been previously mentioned as examples ofprinted report formats that the current invention emulates to takeadvantage of trained operator knowledge and experience.

In addition to waveform representations, the display may also providefor the display of additional information, such as a cardiologist'scomments and reports, for example. Such information may be displayed inone or more separate “windows” located on the display, which windows maybe opened or closed, re-sized, and re-positioned in response to userinput. The interactive display may support measurements in a variety ofdimensions, including those represented by horizontal and verticaldisplay axes.

Time series data that is displayed by an interactive display inaccordance with the principles of the present invention may be obtainedfrom a variety of sources, including stored digitized ECG data, datarepresenting scanned, digitized ECG paper records, or data received,directly or indirectly, and in some sense “live,” from an ECG device.The system is capable of capturing multiple records from differentsubjects, along with relationships among the recordings. Time-seriesdata such as may be displayed and measured through the use of aninteractive display in accordance with the principles of the presentinvention may be organized generally so that recordings from a patientin a single sitting is referred to as a session, and recordings over aperiod where experimental conditions are static will be referred toherein as a session.

Continuous, evenly-sampled data from a set of channels may be referredto as an epoch. For example, in a twelve-lead ECG data obtained from sixleads, followed by data from six other leads, the data is organized astwo epochs of a single session. Data from a session may be organized ina packet that includes a header identifying the person from who therecording was obtained, what equipment was used, and when the recordingsbegan. Each epoch may identify data collection characteristics, thecharacteristics of the channels, and annotations that apply across allchannels. An epoch's data collection characteristics include when thedata collection started relative to the session's baseline data and timeand the sample rate for data on each channel, for example. Channelcharacteristics may include the number of bits, zero-offset, units andscale factor by which data is to be multiplied for conversion to givenunits. Filtering information, such as characteristics of bandpass,highpass, and lowpass, for example, may also be included.

Annotations at the session level can be used to indicate such events aswhen study procedures were performed relative to the recording session.Annotations at the epoch level may be used to mark features visible inthe data for multiple channels, e.g., PVCs or periods of A-V block.Annotations at the channel level would typically be used to mark eventsspecific to that channel's data, for example, the beginning of the Pwave. Each annotation may characterize a particular time point, denotethe beginning and end of an interval, or may be associated with where aparticular amplitude measurement is made. An interactive display inaccordance with the principles of the present invention may support anyof the following ECG-related measurements: duration of all or a selectedpart of a phase of the cardiac cycle, the amplitude of particularfeatures in a cardiac cycle (absolute value or with respect to areference iso-electric line or point), and the duration or presence ofperiods of particular interest where notable events are taking place.Special measurements may include a requirement to determine a minimum orcertain number of annotation points in a certain specified lead or leadsso that particular protocol requirements can be met. The currentinvention may be able to provide a display of these requirements tofacilitate the efficient processing of these requirements.

The new interactive display may produce a report that includes: adigitized twelve-lead ECG waveform, patient demographics, and themeasurements and findings that are related to the analysis of the test.Additionally, the interactive display may, in response to a user promptor as a default, produce and display reduced data. For example, thedisplay may determine and display the average heart rate during a test,the longest PR interval associated with any lead, the longest QRSduration from any lead, or the longest QT interval from any lead.Additionally, the display system may produce and/or display the ratecorrected QT interval based on the QT and the average heart rate—using aselectable conversion methodology, or the frontal plane QRS axisdetermined from combinations of the limb leads. The interactive displaysystem supports study-specific special measurements, such asdetermination of the location of the maximum slope of a particularwaveform feature by calculating the time derivative of the originalsignal.

As described in relation to FIGS. 10 and 11, for example, the system mayalso provide for the display of interpretation codes and statements.Such codes and statements may be automatically generated, using expertsystem, neural network, or other analysis systems, and may be displayedin one or more “windows” within the display. Such windows may be fixedin position, size and other attributes or an operator may resize,relocate, or otherwise alter the window for viewing or comparisonpurposes. The interpretation codes and/or statements may indicate a noteworthy or abnormal ECG feature or finding, and may include an overallassessment code, for example. The system may also provide for thedisplay of additional interpretive comments that may be entered, forexample, by a reviewing cardiologist. Additionally, an overallassessment code, a comparison statement, test identification, summarymeasurements and interpretations may be displayed in a separate textarea. Reports of abnormalities may be the result of computerinterpretation, with cardiologist review. The system may also check theentered interpretation codes and comments for consistency with themeasurements generated by the annotated data and with each other. Thisvalidity checking is provided as guidance to the operator concerningagreed to standards. The system may also provide additional checking toverify that all required special measurements have been completed priorto closing a session.

In an illustrative embodiment, a report file explicitly captures thedata structure of a recording session, epochs comprising data acquiredover the same interval, and representations of data obtained during anepoch. The system may include a mechanism to annotate points in the timeand intervals corresponding to single channels, representations, andsessions. Each report file may include a single unique identifiercreated for each recording, which provides for an unambiguous linkbetween each recording and other data stored for a particular test in astudy.

Pre-set marker information may be extracted from the report file or themeasurement matrix file and used to assist the technician in the initialplacement of the markers. Results of each markup session are stored inthe database. Each time an ECG is reviewed, the most recent set ofmarker data will be used to pre-set the markers. The data from acardiologist's measurements and interpretation will be stored as part ofthe final report concerning the set of data being processed.

The system will use the locations of specifically annotated data pointsto derive or calculate values to be included in the report. Based onstandard electrocardiographic practice or special instructions for aparticular test, the operator will annotate the required points. Allsuch points that contribute to the analysis and interpretation of theECG may be included. The location of the annotated points are withreference to the beginning of the data for that lead and must includethe lead identification for the lead being measured in addition to theidentification label for the point. The invention will support theinclusion of annotation points in addition to those often used instandard analysis of ECGs. In an illustrative embodiment, the intervalsare derived from the indicated data points and will not require aseparate entry. PR, QRS, QT. All intervals may not be available for allleads. Data will only be recorded for those leads in which the intervalswere marked.

In accordance with the principles of the present invention, the systemis able to annotate the presence of particular events of interest,including: Premature beats (by type) to include PVC's, PAC's,Ventricular Tachycardia, for example. An interactive display system inaccordance with the principles of the present invention provides aresolution of at least the sample period of the source data to determinethe location of the points and intervals. Higher resolution may also beprovided to allow estimates or calculations of annotation points to bemade between actual sample values. Measurements may be made across morethan one lead for ECGs that contain synchronous (simultaneouslyacquired) data. These measurements are classified as global in natureand may be used to help identify longest and shortest intervals for aset of leads.

Calculations may be made by the system for the intervals as the intervalend points are indicated. The measurements will be updated each time oneof the markers involved in a measurement is moved. Measurements for theselected beat and for the average of the beats in the selected lead willbe updated as markers are placed or moved. Calculations may be made forthe following intervals. Leads without indicated points will not beincluded in the calculation of the intervals. PR interval—P onset to QRSonset, QRS duration—Q (QRS) onset to J point (QRS end), QT—Q onset to Tend, Heart Rate—based on the average of consecutive R peaks that areselected in a lead, and QTc—this value will use the QT interval and theaverage R to R interval to determine the QTc value. Options for thegeneration of the QTc value will be the use of the Bazeft formula(square root correction), Fredericia formula (cube root correction), theformula using the 2.5 root correction or linear correction, or otherformulae that may be required for rate correction of the QT interval.The minimum and maximum heart rate will also be recorded to allow thedisplay of the rate range information.

In the present invention a variety of statistical methods may be used tocombine the individually determined measurements to determine arepresentative value for the measurement as the summary value for thetest. In an illustrative embodiment, if more than one interval isdetermined in a particular lead, then the average of the values in alead will be used as the value for that lead. If more than one leadcontains values for a particular measurement, then the maximum value forall the leads may be used.

Special measurements may be processed manually or with computationalsupport. In an illustrative embodiment, support for these measurementsis made available by making the database tables to which these valuesmust be entered available to the on-screen measurements program to read.Additionally, the system allows an operator to read from study-specificspecial measurements table, the labeling requirements, the database,table, field, and format. In an illustrative embodiment, undesignatedlabels are available to allow making special measurements andannotations. The labels and the associated database, table, field, andformat information with the instructions for making the measurements maybe automatically or manually loaded based on information provided withthe test. These labels will be used to identify the measurement or eventmarks in the same way as the standard data points and intervals areidentified. The instructions for a particular measurement will beavailable under the general help function or will be displayed when thatspecial measurement mark is selected and the Help button is pressed.This allows operators who are familiar with the measurement to not havescreen space taken up with the instructions.

As previously described, the system supports viewing comparison ECGs. Anoperator may select a comparison ECG (when one or more comparisons areavailable) and view the associated report text and waveforms. The usermay not make any changes to the comparison ECG. The ECG report text willnot be updated with the results of the processing. The details of whatwas changed, when it was changed, and who made the changes will berecorded in a database as part of the tracking of processing of thetest. In an illustrative embodiment, the session records and resultsfile records are an integral part of the database and are available tobe exported.

Output to database tables may include the output from the measurementmarkup portion of the interactive display engine, including allmeasurement points with the label for each point or interval, the leadin which the measurement was made, the location of the point, inmilliseconds, from the beginning of the lead, and the end point, inmilliseconds, for an interval. For global measurement points orintervals, the lead designation will designate the lead group with whichthe markers are related. Also included in the database is the login IDof the person making the changes. Current changes will not obscure therecord of previous changes made to the file. All changes, includingchanges back to original values will be retained. The options andsettings that were used during the changes, including the maximummagnification used, the waveform plot and marker line widths (finalsettings), and the marker color set will be recorded. The followingitems will also be recorded: the date and time when the session wasbegun and when it ended, the lead length of each lead and the offsetfrom the earliest lead, (a zero entry would indicate that no offset ispresent and the lead so indicated is synchronous with the other leadwith an offset of 0), whether a filter was used, and the type of filter,if the data was filtered. This information may be supplied in differentways depending on the procedures used to process the test.

A software implementation of the above described embodiment(s) maycomprise a series of computer instructions either fixed on a tangiblemedium, such as a computer readable media, e.g. diskette, CD-ROM, ROM,or fixed disc, or transmittable to a computer system, via a modem orother interface device, such as communications adapter connected to thenetwork over a medium. Medium can be either a tangible medium, includingbut not limited to, digital or analog communications lines, or may beimplemented with wireless techniques, including but not limited tomicrowave, infrared or other transmission techniques. The series ofcomputer instructions embodies all or part of the functionalitypreviously described herein with respect to the invention. Those skilledin the art will appreciate that such computer instructions can bewritten in a number of programming languages for use with many computerarchitectures or operating systems. Further, such instructions may bestored using any memory technology, present or future, including, butnot limited to, semiconductor, magnetic, optical or other memorydevices, or transmitted using any communications technology, present orfuture, including but not limited to optical, infrared, microwave, orother transmission technologies. It is contemplated that such a computerprogram product may be distributed as a removable media withaccompanying printed or electronic documentation, e.g., shrink wrappedsoftware, preloaded with a computer system, e.g., on system ROM or fixeddisc, or distributed from a server or electronic bulletin board over anetwork, e.g., the Internet or World Wide Web.

Although various exemplary embodiments of the invention have beendisclosed, it will be apparent to those skilled in the art that variouschanges and modifications can be made which will achieve some of theadvantages of the invention without departing from the spirit and scopeof the invention. It will be apparent to those reasonably skilled in theart that other components performing the same functions may be suitablysubstituted. Further, the methods of the invention may be achieved ineither all software implementations, using the appropriate object orprocessor instructions, or in hybrid implementations that utilize acombination of hardware logic, software logic and/or firmware to achievethe same results. Processes illustrated through the use of flow chartsmay not be strictly linear processes and alternative flows may beimplemented within the scope of the invention. The specificconfiguration of logic and/or instructions utilized to achieve aparticular function, as well as other modifications to the inventiveconcept are intended to be covered by the appended claims.

The foregoing description of specific embodiments of the invention hasbeen presented for the purposes of illustration and description. It isnot intended to be exhaustive or to limit the invention to the preciseforms disclosed, and many modifications and variations are possible inlight of the above teachings. The embodiments were chosen and describedto best explain the principles of the invention and its practicalapplication, and to thereby enable others skilled in the art to bestutilize the invention. It is intended that the scope of the invention belimited only by the claims appended hereto.

1. A method of marking time-series medical data that are graphicallydisplayed in the form of a set of waveform traces on a display monitor,each trace embodying a plurality of features, the method comprising thesteps of: A. manually identifying a location of at least one feature ofa first trace; B. displaying on the monitor in relation to theidentified first trace feature location at least one marker; C.displaying on the monitor at least one label corresponding to the markerand the trace feature; and D. automatically displaying on the monitormarkers and labels associated with the one feature on subsequentwaveform traces.
 2. The method of claim 1 wherein the time seriesmedical data is electrocardiogram (ECG) data and and step A comprisesindicating a type of ECG sequence for which a trace feature is to belocated.
 3. The method of claim 2, wherein said type of sequence is aPQJT sequence.
 4. The method of claim 3, wherein step C comprisesdisplaying a label “P” in proximity to a corresponding first marker inthe PQJT sequence.
 5. The method of claim 2, wherein step A comprisesinputting data to a graphical user interface.
 6. The method of claim 2,wherein said type of seuence is a QPJT sequence.
 7. The method of claim6, wherein step C comprises displaying a label “Q” in proximity to acorresponding first marker in the QPJT sequence.
 8. The method of claim2, wherein step A comprises indicating that a QRS onset sequence is tobe labeled.
 9. The method of claim 8, wherein step C comprisesdisplaying a Q onset label in proximity to a corresponding first markerin the QRS onset sequence.
 10. The method of claim 9, further comprisingthe steps of: displaying on the monitor a second marker in relation to aP onset; and displaying a P onset label corresponding to the secondmarker.
 11. The method of claim 1, further comprising the step ofdisplaying a reference line between the at least one marker and thelabel.
 12. The method of claim 1 further comprising the step ofdisplaying the time series medical data as a waveform trace superimposedon a gridded background.
 13. A method of marking time-serieselectrocardiogram (ECG) data comprising the steps of: A. displaying on adisplay monitor the time series ECG data as a waveform tracesuperimposed on a gridded background; B. accepting user input thatindicates a type of (ECG) sequence for which a feature is to be located;C. locating a displayed trace feature of the indicated ECG sequence onthe monitor; D. in response to said user input, placing a first markeron the displayed waveform trace in relation to the located displayedtrace feature; E. displaying a label on the displayed waveform tracethat identifies the first marker; and F. automatically displaying on themonitor markers and labels associated with the one feature on subsequentwaveform traces.
 14. The method of claim 13 further comprising the stepof displaying the gridded background as rectangular cells that delimit40 millisecond by 0.1 millivolt sections of grid-space and displaying aset of super-cells that each delimit 200 milliseconds by 0.5 millivoltsof grid-space.
 15. An apparatus for marking time-series medicalcomprising: A. a display; B. a graphical user interface; and C. acontroller coupled to the display and graphical user interface, saidcontroller configured to: 1) render on the display the medicaltime-series data as at least one trace embodying a set of tracefeatures; 2) locate a displayed medical trace feature; 3) place a firstmarker related to the displayed medical trace feature on the display; 4)place a label on the display that identifies the placed first marker;and 5) automatically display markers and labels that correspond to thedisplayed medical trace feature on subsequent medical traces.
 16. Theapparatus of claim 15, wherein the controller is further configured toaccept a user input that indicates a type of electrocardiogram (ECG)sequence for which a feature is to be located.
 17. The apparatus ofclaim 16, wherein the controller is further configured to accept aninput from a user, the input indicating that a PQJT sequence is to belabeled.
 18. The apparatus of claim 16, wherein the controller isfurther configured to accept an input from a user, the input indicatingthat a QPJT sequence is to be labeled.
 19. The apparatus of claim 16wherein the controller is further configured to accept a user input thatindicates that a QRS onset is to be labeled.
 20. The apparatus of claim15, wherein the controller is further configured to accept a user inputfrom the graphical user interface.
 21. The apparatus of claim 15,wherein the controller is configured to display time series medical dataas a trace superimposed on a gridded background.
 22. An apparatus formarking time-series medical data comprising: A. a display; B. agraphieal user interface; and C. a controller counled to the display andgraphical user interface, said controller configured to: 1) display timeseries medical data as at least one trace embodying a set of tracefeatures superimposed on a gridded background; 2) accept user input thatindicates a type of electrocardiogram (ECG) sequence for which a featureis to be located; 3) locate a displayed medical trace feature; 4) placea first marker on the display related to the displayed medical tracefeature; 5) place a label on the display that identifies the placedfirst marker; and 6) automatically display markers and labels thatcorrespond to the displayed medical trace feature on subsequent medicaltraces.
 23. The apparatus of claim 22, wherein the controller is furtherconfigured to display the gridded background as rectangular cells thatdelimit 40 millisecond by 0.1 millivolt sections of grid-space and todisplay a set of super-cells that each delimit 200 milliseconds by 0.5millivolts of grid-space.
 24. A method for generating a plurality ofmarkers identifying features of time-series medical data that aregraphically displayed in the form of a set of waveform traces on adisplay monitor, to facilitate analysis of the waveform data, the methodcomprising the steps of: selecting a feature marker sequence that eitherrequires: A. manual placement of all feature points to be marked; B.manual placement of an initial marker point and automatic placement ofthe remaining marker points; interactively placing the markers on thedisplayed waveform traces, in the sequence selected in the selectingstep, on one or more of the waveform traces; and in response toplacement of each marker in the selected sequence, automaticallygenerating a label associated with the sequence and displaying thegenerated label in the vicinity of the associated mark on the displayedwaveform.
 25. A method as recited in claim 24, wherein the time seriesmedical data are electrocardiogram (ECG) data, and further comprisingthe step of: interactively adjusting the location of the marker on thedisplay in accordance with operator identification of certain patternsin the ECG.
 26. A method as recited in claim 25, wherein the selectingstep comprises selecting a sequence that requires manual placement of aninitial marker point and automatic placement of the remaining markerpoints, and further comprises: adjusting the location on the display ofthe initial marker point following initial placement thereof.
 27. Amethod as recited in claim 26, wherein the automatic placement ofremaining marker points is based on precalculated interval measurementdata.
 28. A method as recited in claim 26, wherein the initial markerpoint corresponds to a QRS onset point of a portion of the ECG data andthe remaining marker points corresponding to a P wave onset point, theQRS end point and a T wave end point of the ECG portion.
 29. A method asrecited in claim 28, further comprising automatically placing markersand associated labels in subsequent displayed portions of the ECG data,corresponding to previous marker features.
 30. A method as recited inclaim 28, wherein the ECG waveform data is displayed on an interactivedisplay superimposed on a millimeter grid background.
 31. A method asrecited in claim 30, further comprising the step of automaticallyresponding to operator input to magnify the waveform and grid backgroundmaintaining the aspect ratio and size relationships of the waveform andthe grid.
 32. A method as recited in claim 30, wherein movement of themarkers in response to operator input are moved in pixel incrementscorresponding to a time increment represented by the pixel size on thewaveform display.
 33. An apparatus for interactively marking time-seriesmedical data comprising: a graphical display; a storage device; aninteractive operator input pointing device; and a controller coupled tothe graphical display, the storage device, and the interactive operatorinput device, the controller configured to: render the time-series datato display at least one waveform tracing that embodies a set of tracefeatures, respond to operator input to select a specific markersequence, respond to the pointing device to indicate the location offeatures on the display and to place a marker on the waveform based on alocation indicated by the pointing device, automatically place a labelwith the marker, the label corresponding to the next marker in theselected sequence to be placed, respond to operator input to adjust themarker location, and respond to additional operator input to manually orautomatically place additional markers based on a selected markersequence.
 34. Apparatus as recited in claim 33, wherein the time-seriesmedical data is electrocardiogram (ECG) data, and the selected markersequence comprises: an initial marker in the sequence placed inaccordance with manual input and the remaining markers in the sequenceplaced automatically based on data read in by the controller or storedin the controller as a result of previously placed marker sets. 35.Apparatus as recited in claim 34, wherein the locations and labels ofthe markers correspond to calculated ECG waveform intervals, calculatedinterval values being automatically updated when any of the associatedmarkers are moved.